Heterogeneous connections induce oscillations in large scale networks 
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Realistic large-scale networks display an heterogeneous distribution ol connectivity weights, that 
might also randomly vary in time. We show that depending on the level of heterogeneity in the 
connectivity coefficients, different qualitative macroscopic and microscopic regimes emerge. We evi- 
dence in particular generic transitions from stationary to perfectly periodic phase-locked regimes as 
the disorder parameter is increased, both in a simple model treated analytically and in a biologically 
relevant network made of excitable cells. 
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Realistic networks modeling physical or biological sys- 
tems generally display highly heterogeneous connec- 
tions pp. A paradigmatic example is given by the dis- 
tribution of synaptic weights in the central nervous sys- 
tem [2J|3]: 

(i) the precise number of receptors and the extremely 
slow plasticity mechanisms induce a static disorder 
termed quenched synaptic heterogeneities. 

(ii) Thermal noise, channel noise and the intrinsically 
noisy mechanisms of release and binding of neurotrans- 
mitter [4 result in stochastic variations of the synaptic 
weights termed stochastic synaptic noise. 
Experimental studies of cortical areas [5] showed that the 
degree of heterogeneity in the connections significantly 
impacts the input-output function, rhythmicity and syn- 
chrony. Yet, qualitative effects induced by connection 
heterogeneities are still poorly understood theoretically. 
One notable exception is the work of Sompolinsky and 
collaborators [BJ. In the thermodynamic limit of a one- 
population firing-rate neuronal network with synaptic 
weights modeled as centered independent Gaussian ran- 
dom variables, they evidenced a phase transition between 
a stationary and a chaotic regime as the disorder is in- 
creased. Besides stationary and chaotic regimes, synchro- 
nized oscillations is a highly relevant macroscopic state. 
It is related to fundamental cortical functions such as 
memory, attention, sleep and consciousness, and its im- 
pairments relate to serious pathologies such as epilepsy 
or Parkinson's disease [7J |S] • 

In this Letter, we show that both quenched and 
stochastic heterogeneities induce the emergence of 
macroscopic, perfectly periodic oscillations in the mean- 
field limit, corresponding to phase- locked oscillations of 
all neurons in the network. A generic transition from a 
stationary to a periodic regime as disorder increases will 
first be demonstrated in the case of firing-rate neurons. 
Similar reasoning will lead us to uncover the same transi- 
tion in a more realistic network featuring excitable cells. 

Models of cortical areas involve at least two neural 
populations, one excitatory corresponding e.g. to pyra- 



midal neurons and one inhibitory population modeling 
interneurons. In the firing-rate model with N neurons 
and P populations, the dynamics of the membrane po- 
tential (V, i € {1 • • • N}) of all neurons in the network is 
given by the system equations: 
V* N 

v i = ~— + E J « S W) + * e {l • • • JV} 

t p. 3= i 

where pi is the population index of neuron i, r Pi is a 
common characteristic time of all neurons of population 
Pi and I Pi (t) their deterministic input. The interaction 
with the other neurons in the network is given by 
the sum of a synaptic efficacy Jij multiplied by the 
firing rate of the presynaptic cell, a positive sigmoidal 
transform (S(-)) of the voltage. Network heterogeneities 
manifest through random variations of the synaptic 
coefficients between neuron i and j. The two types of 
heterogeneities in the connections are modeled as follows: 

(i) quenched synaptic heterogeneities: the Jij 
are independent random variables with law 
]\[{J PiP JN p . , a j y/N Pj ), where N a denotes the number 
of neurons in population a G { 1 • • • P} . 

(ii) stochastic synaptic noise: Jij are stochastic pro- 
cesses, e.g. only depending on i and pj and given 
by Jij — JpiPj/Np. + cr/N Pj Wl Pj where the collection 
(W^ a ,k = 1 • • ■ N, a = 1 ■ ■ ■ P) are independent Gaus- 
sian white noise. 

In both case, the scaling ensures that the disorder in 
the system is quantified by the parameter a and the ef- 
fective input by the coefficients (J a p,a,(3 € {1---P}). 
These two models lead to relatively different network 
equations: model (i) corresponds to a system of N cou- 
pled ODEs with random coefficients, whereas model (ii) 
corresponds to SDEs. Incidentally, the analysis of model 
(ii) will be analytically simpler and give great insight on 
the dynamics of system (i). 

In the stochastic synaptic noise case, the network sat- 
isfies the system of Langevin ltd equation: 
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These equations are evocative of the kinetic theory of 
gases, and the mean-field theory [9] allows rigorously 
demonstrating, in the limit where all the Np tend to in- 
finity, the propagation of chaos property and the conver- 
gence in law of V? for pi = a towards V t a solution of the 
well-posed system of P mean-field equations: 
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where W"^ are independent white noise and E[-] denotes 
the statistical expectation. Using an implicit form of this 
equation given by the variation of constant formula, we 
prove that this system admits Gaussian solutions attract- 
ing exponentially fast every solution. A striking property 
is that the mean and variance of this Gaussian solution 
satisfy a closed system of ODEs. In details, noting \i a 
the mean of the Gaussian solution related to population 
a, v a its variance and /(/J., v) the expectation of S(X) 
for X ~ N{n, v) [TU], we have: 
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The very complex stochastic mean-field equation thus 
exactly reduces to a mere system of coupled nonlinear 
ODEs characterizing the permanent regime. Moreover, 
the level of disorder appears as a parameter, which allows 
precise identification of noise-induced transitions using 
bifurcation theory. 

The bifurcation diagram of these mean-field equations 
in an excitatory/inhibitory case with constant input I a 
shows a very intricate structure. As a function of a and 
the external current I\ (Fig. [I]), it includes local bifur- 
cations (Hopf, fold, Bogdanov-Takens, cusp and Bautin) 
and global bifurcations (homoclinic and fold of limit cy- 
cles) induced by the disorder, defining 11 regions for a 
in which the system responds in the same manner to the 
external input I\ [llj . In particular, there exists an inter- 
val of I\ for which the disorder induces transitions from 
a stationary regime to stable periodic orbits (upper-right 
inset of Fig. [I]). When increasing a, these limit cycles 
appear through a saddle-homoclinic bifurcation and dis- 
appear through a Hopf bifurcation. 

This analysis proves that the solution of the stochas- 
tic mean-field equations is perfectly periodic in law in 
a specific interval I of the disorder parameter, and sta- 
tionary for smaller or larger heterogeneity. From the mi- 
croscopic viewpoint, the propagation of chaos property 
ensures that all neurons in a given population have the 
same law, and hence present phase-locked oscillations for 
disorder levels in X (see Fig. [2]). 

With this example in mind, we now turn to the 
quenched heterogeneities case. In the thermodynamic 
limit, using the formalism developed in the dynamical 
mean-field theory for spin-glasses [T31 [H] , the solution of 
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FIG. 1: Bifurcation diagram of the moment equations in a 
and h. Jn = 15, J 12 = —12, J 2 i = 16, J22 = —5, I 2 = —3 
following [12], T a = 1, S(x) = erf(x). Black region: 3 fixed 
points, Gray: cycles. Vertical bands: 11 typical regimes (as 
I\ is left free) depending on a . Blue: saddle-node, pink: 
Hopf, plain (resp. dashed) green: saddle (resp saddle-node)- 
homoclinic, orange: fold of limit cycles (FLC). BT: Bogdanov 
Takens. CP: Cusp. GH: Bautin. Upper-right inset shows 
disorder-induced oscillations for I\ — 0: Plain (dashed) blue: 
stable (unstable) equilibria, pink: periodic orbits, LP: saddle- 
node, H: Hopf. 

the network equations converge towards the solution of 
the non-Markovian P implicit well-posed equation: 
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In these equations appear the effective interaction pro- 
cesses {U^p{t)) a encapsulating the randomness of the 
synaptic coefficients. These are independent Gaus- 
sian processes with mean J a pE{S(vf)] and covariance 
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Similar to the stochastic synaptic 
noise case, due to the intrinsic linear dynamics, any solu- 
tion of ^ is exponentially fast attracted towards Gaus- 
sian solutions. The mean of Gaussian solutions satisfies 
exactly the same equation as in the previous case (equa- 
tion 0). Unfortunately in the quenched heterogeneity 
case, we cannot describe the variance dynamics through a 
simple ODE. It depends on the covariance of V a through 
the implicit integral equation: 

C a (t,s) = e -( t+s ^ \e 2t »/ T *C a (t ,t ) + 
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The dependence of C a (t, s) on the whole past of the so- 
lution is fundamentally related to the non-Markovian na- 
ture of the equations. This more complicated framework 
does not allow analytical study of the equilibria and their 
stability. 



3 



However, the fact that the solutions of the mean-field 
equations, for both stochastic and quenched synaptic het- 
erogenetities, have a mean characterized by the same 
equation suggests that quenched heterogeneity-induced 
oscillations may arise. And indeed, it is easy show that 
the variance of the solution v a (t) = C a (t,t) is an in- 
creasing function of the disorder parameter a. Artifi- 
cially considering the variance as a parameter in equa- 
tion (JlJ, routine bifurcation analysis shows that the ex- 
citatory/inhibitory system considered undergoes, as v is 
increased, a saddle-homoclinic bifurcation yielding oscil- 
lations that disappear through a Hopf bifurcation [T5] , 
Increasing the disorder parameter in the system ^ will 
hence lead the variance to exceed the value related to the 
saddle-homoclinic bifurcation, yielding a transition from 
stationary to periodic solutions, that correspond to syn- 
chronized oscillations at the microscopic level because of 
the identical distribution of all neurons in the thermody- 
namic limit. The disorder-induced transition is precisely 
observed in the numerical simulations of large networks 
as displayed in Fig. [2] Two transitions, first from sta- 
tionary solutions (weak heterogeneity) to synchronized 
oscillations and then back to a stationary regime, are 
observed as heterogeneity increases. 




FIG. 2: Network dynamics in the stochastic noise case (left) 
and quenched heterogeneity case (right), for I\ = and in- 
creasing values of the disorder parameter a : (left): 0.5, 1.5 
and 6, (right): 0.9, 1.6 and 3. 5 000 neurons per population, 
10 sample trajectories (red: population 1 and blue: popula- 
tion 2) and macroscopic statistics (black: population 1 and 
cyan: population 2). 

Beyond the striking statistical similarities, the very dif- 
ferent underlying microscopic settings of stochastic and 
quenched heterogeneity correspond to distinctive quali- 
tative features. In particular, when periodic solutions 
lose stability, the synaptic noise networks displays a pure 
stochastic dynamics whereas in the quenched heterogene- 



ity case the network displays a chaotic dynamics, pre- 
cisely related to the same phenomenon as in [6] which 
may be explained through the analysis of the eigenvalues 
of the random connectivity matrix. 

One clear advantage of firing-rate models are their rel- 
ative simplicity, allowing theoretical approaches to un- 
cover the phenomenon of disorder-induced transitions. A 
key ingredient allowing the analytical developments was 
the linearity of the intrinsic neuronal dynamics, yielding 
Gaussian solutions to the mean-field equations. Though 
valid to reproduce a large body of experimental results, 
this linearity neglects the prominent excitable nature of 
neurons. We now show that the same transition occurs in 
a realistic one population network composed of Fitzhugh- 
Nagumo (FN) neurons interconnected with random elec- 
trical synapses. This model describes the dynamics of 
neuron i through its membrane potential v l and a recov- 
ery variable w l : 

v\ = f{v\) - W \ + E^Li JiM - 4) + i 

w\ = a{bv\-w l t ). 

In this equation, a represents the time-scale of the re- 
covery variable, f(v) = v(l — v){v — k) and / is a de- 
terministic input. We choose parameters corresponding 
to the case where FN neurons present a unique stable 
equilibrium (v*,w*), and consider the synaptic hetero- 
geneity case with Gaussian weights of positive mean J 
and standard deviation a. In that (.'MSG, M trivial solu- 
tion of the network equations (whatever J^) is given by 
(v t ,w i ) = (v*,w*) for all i. This fixed point is stable for 
weak heterogeneity (see Fig. |3| . As the disorder param- 
eter a is increased above a critical value, this stationary 
regime loses stability and periodic synchronized oscilla- 
tions appear. As the disorder is further increased, these 
oscillations progressively disappear in favor of a chaotic 
regime of individual neurons in which the empirical mean 
becomes stationary. Analytical studies in that case are 
way harder than in the firing-rate case. Moment expan- 
sion and truncation may allow approximating the ran- 
dom system into a deterministic system of ODEs |16j . 
Motivated by the previous study, we propose to approx- 
imate the solutions by Gaussian processes with averaged 
voltage fj, v and recovery variable fj, w and assuming con- 
stant standard deviation A on the voltage. Under this 
approximation, the variables satisfy the ODE: 

I At> = f(l*v) + ^ 2 (! + K + 3jU„) - fi w + I 
\(i w = a(b^ v - 

and this system shows a transition towards oscillations as 
A is increased (occurring when a is increased), as plot- 
ted in Fig. [3] (d). This transition towards oscillations 
in the Gaussian approximation allows accounting for the 
heterogeneity-induced transition to oscillations observed 
of the original network. 

We hence showed that perfectly periodic phased-locked 
oscillations arose in the mean-field limit of large scale net- 
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FIG. 3: FN network, a = 0.4, 6 = 2,/ = 0.5, J = 1.5, « = 2. 
20 trajectories of the voltage chosen among 5 000 neurons 
(pink) (thick black: one sample trajectory), (a) weak het- 
erogeneities (a = 0.5): "deterministic" behavior, (b) interme- 
diate (a = 1): synchronized oscillations and (c) strong hetero- 
geneities (a = 1): chaotic behavior, (d) bifurcation diagram 
of system Q: (A): stationary solutions, (B): bistability, (C): 
oscillations. 



the mean, after febrile seizures in developing rats [3]. 
Several intriguing questions remain unanswered. First 
is the heuristic understanding of the systematic appari- 
tion of the oscillations in the presence of heterogeneities. 
Second is the striking parallel between stochastic net- 
works and randomly connected quenched networks. Be- 
sides this particular transition from stationary to peri- 
odic solutions, a number of more complex bifurcations 
were presented here in the synaptic noise case and a tran- 
sition to chaotic individual trajectories exhibited in the 
quenched case and will be examined further and may 
provide deeper understanding of such heterogeneous net- 
works. Eventually, the dynamics of such networks with 
correlated synaptic weights, or with a non-recurrent con- 
nectivity graph, are deep questions that are still largely 
left unexplored. 
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works, induced by the presence of heterogeneous connec- 
tivity weights. Because of the large amplitude of the os- 
cillations (appearing through homoclinic or after a fold of 
limit cycle), finite-size networks, converging in 0(l/v~N) 
towards the mean-field equations, will always display the 
same kind of perfectly periodic synchronized oscillations 
as soon as l/y/N is small compared to the amplitude of 
the cycle. Both nonlinearities and network structure are 
implicated in this phenomenon: nonlinearity couple the 
dynamics of the ensemble average to the degree of hetero- 
geneity, and negative feedback loops (inhibitory neural 
population in the firing-rate model and adaptation vari- 
able in the FN model). This phenomenon is distinct of 
more usual noise-induced transitions studied in the field 
of nonlinear systems with noise ([13 [18] and references 
therein) . It distinguishes from stochastic resonance since 
there is no periodic input, and from coherence resonance 
since here the mean and standard deviation are perfectly 
periodic with phase determined by the initial condition. 
But the main difference with these previous works is the 
fact that all these phenomena were identified for external 
noise whereas here the phenomenon evidenced is related 
to the level of heterogeneities. In the quenched hetero- 
geneity case, outstanding works addressed the problem of 
chaotic or order-disorder transitions [6| 119] or the dynam- 
ics of quenched randomly in coupled oscillators [20] , both 
differing from the transition reported here since we do not 
deal with coupled oscillators, and addressed a transition 
to synchronized oscillations. 

A sudden apparition of synchronized large-amplitude ar- 
bitrarily slow oscillations is strongly evocative of epileptic 
seizures. These findings provide a theoretical explanation 
to the recent experimental results that reported persis- 
tently increased synaptic variability, without changes in 
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